%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '24px'}, 'flowchart': {'nodeSpacing': 80, 'rankSpacing': 80, 'htmlLabels': true, 'useMaxWidth': false, 'padding': 8}}}%%
flowchart TB
A["Exploratory Research<br>Methods"] --> B["Direct<br>(Nondisguised)"]
A --> C["Indirect<br>(Disguised)"]
B --> D["In-Depth<br>Interviews"]
B --> E["Focus<br>Groups"]
C --> F["Projective<br>Techniques"]
style B fill:#fff,stroke:#000
style C fill:#1a4480,stroke:#1a4480,color:#fff
style A fill:#fff,stroke:#000
style D fill:#fff,stroke:#000
style E fill:#fff,stroke:#000
style F fill:#fff,stroke:#000
The power of the mask
“Man is least himself when he talks in his own person. Give him a mask, and he will tell you the truth.”
– Oscar Wilde
Projective Techniques
Projective technique: A form of questioning that encourages respondents to project their underlying motivations, beliefs, attitudes, or feelings regarding the issues of concern…
Onto a third party
Into a task situation
Onto an inanimate object or other vague stimulus
Can be used within focus groups or in-depth interviews:
Role playing
Word association
Sentence completion
Cartoon (balloon)
Why Projective Techniques?
What do you think?
Direct questioning is sometimes of limited value because people may not be aware of the real reasons for their feelings or choices, and resort to obvious or conventional explanations.
Indirect approaches, such as projective techniques, are aimed at uncovering “subconscious” motives or helping people voice hard-to-articulate (or sensitive) ideas
Role playing
Role playing: Respondents are asked to play the role or assume the behavior of someone else.
Classic Example: Nescafe (1950)
Context: unexpected customer resistance to Nescafe instant coffee, marketed as a new easy way to make coffee at home
When women were questioned directly about why they did not like instant coffee, the typical answer was they did not like its flavor
Researchers suspected flavor was a cover story
Experiment by Mason Haire [Journal of Marketing, Vol. 14, No. 5 (Apr. 1950), pp. 649-656]
The Nescafe Shopping List Study
100 housewives (it was the 1950s…) were asked to review one of two shopping lists and to role-play as a shopper going to the grocery store to buy the items on the list.
Shopping List 1
2 pounds of hamburger
Bunch of carrots
1 can, Rumford’s Baking Powder
Nescafe Instant Coffee
2 cans, Del Monte Peaches
5 pounds of potatoes
Shopping List 2
2 pounds of hamburger
Bunch of carrots
1 can, Rumford’s Baking Powder
Maxwell House Coffee
2 cans, Del Monte Peaches
5 pounds of potatoes
This shopper is:
Lazy (48%)
Did not plan purchases (48%)
Thrifty (4%)
Good wife (4%)
This shopper is:
Lazy (4%)
Did not plan purchases (12%)
Thrifty (16%)
Good wife (16%)
Nescafe’s Response
Later ads focused less on “quick, efficient, economical”, and instead shifted towards emphasizing social acceptability and quality (“satisfy your coffee hunger”), and portraying instant coffee as something you could “serve to guests with pride”.
Sentence Completion
Sentence completion: A projective technique involving the presentation of incomplete sentences to respondents who are asked to complete them in their own words. The goal is to derive “automatic” connections to stimuli.
Do you think it’s important to give blood?
Most people say yes in direct elicitation (i.e., given a numeric rating scale)
Sentence completion task
Respondent 1:
I always give blood during blood drives at work, unless I’m sick.
People who don’t give blood are pretty selfish in my opinion.
Respondent 2:
I always give blood during blood drives at work, unless I’m in a hurry.
People who don’t give blood just don’t like needles.
Word Association Test (WAT)
A word-association test is a projective technique where participants are presented with a word or phrase and asked to quickly respond with the first word or idea that comes to mind.
The goal is to uncover subconscious or less-filtered attitudes, feelings, and associations that participants hold toward the stimulus, which might not surface through direct questioning.
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '18px', 'background': '#ffffff'}, 'flowchart': {'nodeSpacing': 20, 'rankSpacing': 30}}}%%
flowchart TB
N["NIKE"] --> A1["Sports"]
N --> A2["Shoes"]
N --> A3["Michael<br>Jordan"]
N --> A4["Cool"]
N --> A5["Just<br>Do It"]
style N fill:#fff,stroke:#000,stroke-width:2px
style A1 fill:#fff,stroke:#000
style A2 fill:#fff,stroke:#000
style A3 fill:#fff,stroke:#000
style A4 fill:#fff,stroke:#000
style A5 fill:#fff,stroke:#000
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '18px', 'background': '#ffffff'}, 'flowchart': {'nodeSpacing': 20, 'rankSpacing': 30}}}%%
flowchart TB
O["ORGANIC FOOD"] --> B1["Healthy"]
O --> B2["Sustainable"]
O --> B3["Environmentally<br>friendly"]
O --> B4["Expensive"]
O --> B5["Vegetables"]
style O fill:#fff,stroke:#000,stroke-width:2px
style B1 fill:#fff,stroke:#000
style B2 fill:#fff,stroke:#000
style B3 fill:#fff,stroke:#000
style B4 fill:#fff,stroke:#000
style B5 fill:#fff,stroke:#000
Cartoon (Balloon) Test
A cartoon test is very similar to a sentence completion task and serves much the same purpose. Cartoon tests show a cartoon illustration in which one or more characters are present. At least one of the characters has an empty dialogue bubble.
Third-Person Technique
Third-person technique: The respondent is asked to talk about someone else, such as a neighbor or a friend. In this case, the respondent is asked to relate the beliefs and attitudes of a third person rather than directly expressing personal beliefs and attitudes.
Goal: Minimize the social pressure to give a politically correct response
Examples:
Which candidate do you think your neighbor is voting for? (Trump or Harris)
Compare to “Which candidate are you voting for? (Trump or Harris)”
How much do you think other people your age drink on a Saturday night?
Pros and cons of projective techniques
Pros
Disguising the purpose of the study allows us to elicit responses that subjects may be unwilling or unable to give otherwise
Useful when the issues are personal, sensitive, or subject to strong social norms
Useful when underlying motivations, beliefs, and attitudes are operating at a subconscious level
Cons
Require highly trained interviewers
Risk of interpretation bias
When the researcher unintentionally reads their own assumptions, expectations, or preferred story into participants’ ambiguous responses
Skilled interpreters are also required to analyze the responses
Engage people in unusual behaviors
Descriptive Research
What is descriptive research?
Descriptive research: research targeted at describing the characteristics of an existing marketing situation
Questions related to who, what, where, when, how
Who buys iPhone 17? When do riders use Uber? Where do consumers buy coffee on campus? How optimistic are people about the economy?
Mostly Quantitative
Common data collection approaches:
Observational data (secondary)
Store audits, consumer panels, TV panels, online panels
Surveys and questionnaires (primary)
Observational Data
What is observational data?
Observational data: Observe and record patterns of people, events, or other phenomena
Expedia: Rick spent 20 minutes on Expedia searching for vacation packages. After clicking on the pages of five different hotels, he booked a 3-night stay at Hyatt Zilara in Cancun.
Apple: Zach watched the entirety of the newest season of Slow Horses in one sitting.
Data can be collected by human observers, cameras, RFID, social media platforms, etc.
Common Industry Data
Store audits (retail scanner data)
Consumer panels
Media panels
TV
Internet
Social media
Firm-specific data
Social media profiles
Website clickstream
App behaviors
Internet usage from an ISP
Store audit data
Store audits (retailer scanner data) consist of weekly pricing, volume, and store environment information generated by point-of-sale systems from participating retail chains across markets.
Tens of thousands of participating grocery, drug, mass merchandisers in major US markets
Weekly product data for >2.6 million UPCs (food, drug, liquor, convenience)
units (sales), price, feature indicator, and display indicator
E.g., Supermarket X sold 200 units of Tropicana (59oz) in week 2 at $3.99
Sources: Nielsen, IRI
Example questions:
How does assortment of category Z vary across different store types (grocery vs. drug vs. mass merchandisers)?
What is the effect of in-store displays on sales of product Y?
Does our product get better placement (eye-level shelf position, end cap, etc.) than competitors’ products?
Demand analysis
Competitor intelligence
Consumer panels
Consumer panel: Households provide information about purchases and/or media consumption. The same households are tracked over time.
National Consumer Panel:
Nielsen/IRI Panelists (40,000-60,000)
Demographic info (e.g., income, size, education, age)
Geographic info (county, zip code)
Product ownership (e.g., TV, car)
Weekly purchase and/or viewing info
Example: Household Z purchased Yoplait from Safeway in Broadway Kino plaza in Week 1, Dannon from a CVS in Week 2, Chobani in Week 3 and Chobani in Week 5.
Example questions:
Who buys what, how often, how much, and where?
How do purchase patterns vary by demographics (age, income, family size)?
Do promotions increase purchase frequency, lead to stockpiling behavior, or induce brand switching?
Other media panels
Media panels (Nielsen, iSpot, TVision):
Measure the audience from TVs, computers, and other devices.
Average number of viewers for Super Bowl = 100 M
Clickstream data (ComScore):
Pages a user visits and the sequential stream of clicks as they move across the web
What is the reach and frequency of my ad campaign across demographics?
How much overlap is there between TV and digital ad reach?
Where should we place ads to effectively reach our target audience?
Shopping funnel analysis: Does position of the product on landing page affect the likelihood of purchase?
Social media monitoring
Social media monitoring is a way to observe people and how they interact with each other online and within social media
Can be perceived as “a focus group of millions of people”
Applications:
Millions of “social mentions” of the Fortune 100 companies each month
Access all the mentions (of your brand/product) and aggregate the user-generated content across different types of platforms into a single stream of information
Example questions:
What are people saying about our new product launch?
How do current events beyond our control (e.g., natural disasters, political events) impact brand sentiment?
Cautions: Self-selected sample (are the people posting on platform X representative of the target market?)
Surveys
What is a Survey?
Ask respondents for information using verbal or written questions
Questions are fixed and structured
Respondents are a sample of the researcher’s “target population”
Data are mostly numeric (e.g., multiple choice)
Useful when…
Respondents know the answer
We care what they have to say
They are willing to tell us the truth
Types of Survey Methods
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '16px'}, 'flowchart': {'nodeSpacing': 20, 'rankSpacing': 40}}}%%
flowchart TB
A["Survey Methods"] --> B["Telephone-<br>Administered"]
A --> C["Personal-<br>Administered"]
A --> D["Self-<br>Administered"]
B --> B1["Traditional<br>Telephone"]
B --> B2["Computer-Assisted<br>Telephone Interviewing<br>(CATI)"]
C --> C1["In-Home"]
C --> C2["Mall Intercept"]
C --> C3["Computer-Assisted<br>Personal Interviewing<br>(CAPI)"]
D --> D1["Mail"]
D --> D2["Mail Panel"]
D --> D3["Online"]
D --> D4["Drop-off"]
style A fill:#fff,stroke:#000
style B fill:#fff,stroke:#000
style C fill:#fff,stroke:#000
style D fill:#cce5ff,stroke:#000
style B1 fill:#fff,stroke:#000
style B2 fill:#fff,stroke:#000
style C1 fill:#fff,stroke:#000
style C2 fill:#fff,stroke:#000
style C3 fill:#fff,stroke:#000
style D1 fill:#fff,stroke:#000
style D2 fill:#fff,stroke:#000
style D3 fill:#cce5ff,stroke:#000
style D4 fill:#fff,stroke:#000
In recent years, the vast majority of survey-based marketing research is conducted via online surveys
Low cost, speed, ease of targeting, scalable
But, some populations may be more difficult to reach
Causal relationship: the relationship between cause and effect.
“When you do X, Y will happen”
Correlation is not sufficient
Alternate explanation: Reverse causality
Organic growth in sales leads to larger advertising budget
Alternate explanation: Third factor
Predictable cyclicality in demand + advertising
What product might exhibit this pattern?
Correlation vs. causation
Examples in Marketing:
Starbucks release the pumpkin spice latte in October. Sales of hot drinks in Starbucks start increasing in October.
Does the release of PSLs cause more sales of hot drinks?
Product rankings on Amazon, WalMart, BestBuy
Do higher ranked products sell better, or do products that sell better get higher ranks? Both?
Three Factors Necessary for Causation
1. Correlation (Concomitant Variation)
Evidence of association between X and Y
Can be shown with ρ, the correlation coefficient
2. Temporal Antecedence
X must occur before Y
3. No third factor driving both X and Y
Control of other causal factors
Causation Diagram
A change in X causes a change in Y.
Three Factors Visualized
Correlation
When X changes, Y changes When Y changes, X changes (association)
Antecedence
X comes before Y Y doesn’t come before X
No Third Factor
No third factor, Z, causes the change to both X and Y.
Causation: Combining all 3
Causal Research Designs
Causal Research designs: study the causal effects of changing something in the marketing environment (X) on an outcome of interest (Y).
Independent variable (IV): measures what the researcher changed
Dependent variable (DV): measures of effects or outcomes that occur as a result of changes in levels of independent variable(s)
Experiments, Randomized Control Trials, Test Markets, A/B Testing
The ONLY research designs that rigorously establish causality
Difficult part: making sure that change in DV was not caused by something other than IVs (confounding variables, confounders). I.e., no third factor that affects IVs and DV simultaneously
Experimental Design: Treatment and Control Groups
Treatment group
Group exposed to the manipulation/treatment
X = ORANGE
Control group
Group not exposed to the manipulation/treatment
X = BLUE
Random Assignment
Participants randomly assigned to condition
This avoids selection effects
Balances individual differences among participants across groups
Netflix Design Example
Netflix wants to understand the causal effects of box art (A, B, C, or D) on the probability of clicking on the show.
Experiments differ in their internal and external validity
Internal validity: ability to confidently draw causal conclusions
Basically, how clean is the experiment? Is it free of confounds?
External validity: ability to generalize from research setting to other contexts (i.e., the real world)
Is the research environment so context-free that results are useless?
Difficult to achieve high internal AND external validity
Example: Internal vs. External Validity
Suppose you want to know what tastes better: Coke or Pepsi
Design A: Unlabeled cups
Design B: Labeled cans
Which design has higher internal validity? Which design has higher external validity?
Experiments and Validity: Confounds
A confound (third factor) is a factor that influences both the dependent variable and independent variable.
Example of a confounded experiment: Suppose you want to learn the effect of container shape on amount of soda consumed. You run the following experiment. You take an unlabeled can and fill it with Pepsi. You take an unlabeled bottle and fill it with Coke. You randomly assign participants to drink from either the can or the bottle, and measure how much soda they consume.
What is the DV?
What is the IV?
What is the confound?
Lab vs. Field Experiments
Lab Experiments
Artificial (controlled) setting
Lab’s subject pool
Pro:
Limited effects from extraneous variables
Quick and inexpensive
Con:
Not a natural setting (generalization might be a problem)
→ Higher internal validity
Field Experiments
Natural setting
Supermarkets, online
Pro:
Generalizable
Con:
Expensive and time-consuming
Difficult to control extraneous variables
No secrecy
→ Higher external validity
For Next Class
To Do
Meet with your team
Complete GA1
Choose/finalize a research topic
Agree on norms for group work
Preview GA2
Conduct a focus group
Appendix
How to Be a Better Interviewer: RASA
RASA is a simple framework popularized by sound and communication expert Julian Treasure for cultivating better listening and more effective communication:
Receive - Be fully present. Actively pay attention to what the speaker is saying. Avoid distractions and give them your undivided focus.
Appreciate - Use small verbal (e.g., “I see,” “Mm-hmm”) or nonverbal cues (nodding, eye contact) to let the speaker know you are attentive and value what they’re sharing.
Summarize - Recap or paraphrase what you’ve heard to ensure mutual understanding. Phrases like “So, what I’m hearing is…” help confirm you’ve caught the essence.
Ask - Invite clarification or deeper insight by asking open-ended questions. This both demonstrates engagement and encourages the speaker to elaborate.
Customers who (1) Received coupons and (2) Did not redeem > Customers who did not receive coupons
Is this a good comparison?
Could there be other differences between these groups?
Coupon recipients were targeted
Composition of the sample on the left can be very different from the right
A potential third factor: targeting variables
“In a experiment with eight national retailers, we analyzed campaigns involving more than 500,000 targeted coupons, for items representing more than 300 brands, mailed out over 16 months.”
What is a Good Controlled Experiment?
Create groups where the only difference between them is the experimental “treatment”
e.g., testing the effect of Coupons
Randomized Controlled Trials (RCT)
Treatment Group
Random set of customers are sent coupons
Control Group
Rest of the customers are not sent coupons
Only difference between individuals in (A) and (B) is the coupons!
Social media monitoring
Social media monitoring is a way to observe people and how they interact with each other online and within social media
Applications:
Millions of “social mentions” of the Fortune 100 companies each month
Access all the mentions (of your brand/product) and aggregate the user-generated content across different types of platforms into a single stream of information
Example questions:
Cautions: Self-selected sample (are the people posting on platform X representative of the target market?)